Trust in Numbers: The Pursuit of Objectivity in Science and Public Life
Authors: Theodore M. Porter, Theodore M. Porter
 (Z-Library)_thumbnail.jpg)
Overview
This book explores the complex relationship between objectivity, quantification, and modern life, particularly in science and public affairs. I argue that quantification, while often presented as a purely objective and rational tool, is also a social and political technology used to manage and control, not just to understand. The book’s central argument is that objectivity isn’t a natural state but is constructed through social processes, administrative routines, and technologies of trust. These processes often involve standardizing practices, creating uniform measures, and establishing rules that can be applied across vast distances and diverse contexts, minimizing the need for personal contact or localized knowledge.
The book examines how numbers and quantitative methods have become essential tools for building trust and exercising authority in a range of settings, from scientific laboratories and government bureaucracies to business corporations and academic disciplines. While not opposed to numerical reasoning, I challenge the naive view that quantification automatically guarantees objectivity or eliminates bias. On the contrary, the pursuit of objectivity often reflects an underlying distrust of personal judgment and a desire for impersonal rules that can be applied without discretion. I argue that the very act of quantifying shapes the world it purports to describe, transforming the objects of scientific inquiry and the social categories we use to understand ourselves.
The intended audience is anyone interested in the role of science and expertise in modern society. It addresses scholars in science and technology studies, history of science, sociology of science, and political science, as well as anyone concerned with the ways in which knowledge is produced, validated, and used in the public sphere. It aims to show that quantification is not merely a technical issue, but a cultural and political one with profound implications for how we organize our institutions, make decisions, and understand the world around us. The book’s relevance is even greater now, in the age of ‘big data,’ algorithms, and the expanding influence of quantitative metrics in nearly every aspect of modern life, from education and healthcare to finance and criminal justice. My historical analysis provides a framework for understanding the roots of these developments and for critically evaluating the claims made on behalf of objectivity and quantification.
Book Outline
1. A World of Artifice
Modern science is characterized by a drive for objectivity, but this objectivity is less about truth and more about a quest for impersonality. This impersonality takes the form of standardized measurements, a struggle against subjectivity and personal judgment in an effort to create knowledge that appears independent of the people who make it. While science depends on the transmission of skills and practices between laboratories, standardization plays a key role in making local knowledge generally valid. The spread of instruments and the creation of stable, purified objects for experiments, like standardized reagents and lab mice, transform science from a collection of local research communities to a global network.
Key concept: ‘Quantification is a technology of distance.’ This means that the highly structured, rule-bound language of mathematics allows for communication that transcends local knowledge and personal trust, useful for coordinating activity and resolving disputes across geographical and cultural boundaries. It also minimizes the need for intimate knowledge and personal trust because the rules for collecting and manipulating numbers are widely shared.
2. How Social Numbers Are Made Valid
Numbers are used to measure not just nature, but also society. The quest for objectivity in social measurement faces challenges similar to those encountered in standardizing scientific measurements. While social categories and constructs like “society,” “crime rates,” or “unemployment” seem to offer objective descriptions of social reality, they are also shaped by the social and administrative processes used to define and measure them. Social statistics, then, do not simply reflect social reality, they actively construct and shape it.
Key concept: ‘Social measurement is rarely simply imposed from outside. Instead quantification is implicit “in the social process itself, before any social scientist intrudes.’ This means that quantification isn’t just something imposed on social phenomena by researchers, but is often deeply embedded within social processes themselves.
3. Economic Measurement and the Values of Science
Quantification is not simply a tool for understanding the world, but also for shaping and managing it. This is evident in fields like economics, where measurement and calculation have become central to managing businesses and governments. From cost accounting to cost-benefit analysis, quantitative methods are deeply intertwined with administrative practices. The very act of quantifying and measuring shapes the economic world and its actors.
Key concept: ‘Quantification is a social technology.’ This means quantification is not merely a tool for understanding the world, but also for shaping and managing it, from factory production to government policies.
4. The Political Philosophy of Quantification
The pursuit of objectivity, closely tied to quantification, has become increasingly important in political and administrative decision-making. This stems from the tension between expert judgment and public accountability in democratic societies. When expert judgment is viewed with suspicion, be it due to conflict of interest, lack of transparency or lack of perceived fairness, quantification offers a seemingly impartial alternative. Quantification thus empowers bureaucratic officials, lending them legitimacy through the appearance of making decisions ‘by the numbers.’
Key concept: ‘Objectivity lends authority to officials who have very little of their own.’ Quantification and the pursuit of objectivity have become crucial tools for asserting authority and legitimacy in bureaucratic settings.
5. Experts against Objectivity: Accountants and Actuaries
Objectivity is not an inherent property of accounting but is shaped by context and social organization. When accountants are part of a secure, relatively self-governing profession, they tend to emphasize expert judgment and resist standardization. In contrast, when confronted with public pressure or the oversight of external authorities, they tend to emphasize their adherence to strict rules and procedures.
Key concept: ‘A face to face group has no great need of writing.’ This quote, from Jack Goody, highlights that bureaucratization and the need for record-keeping are driven by a need to deal with distance and lack of personal contact. As organizations and communities grow larger, impersonal record-keeping becomes increasingly essential for coordinating activities and maintaining control.
6. French State Engineers and the Ambiguities of Technocracy
French state engineers, unlike their American counterparts, historically enjoyed a high degree of autonomy and prestige. This stemmed from their close ties to the state and their elite status. Within the French administrative system, expert judgment and nuanced decision-making were valued over rigid, standardized procedures, shaping the way they applied economic analysis to public works projects.
Key concept: ‘Each functionary is superior to every authority insofar as he carries out his function.’ This principle, central to the French administrative system, expresses the idea that officials have absolute control within their domain and should not be subject to external interference. This reflects an emphasis on hierarchy and a preference for internal regulation over public accountability, which can shape the development and application of quantitative methods within bureaucratic organizations.
7. U.S. Army Engineers and the Rise of Cost-Benefit Analysis
In contrast to the French, American Army Corps engineers have faced a much more fragmented and politically charged environment. The Corps has been criticized for being vulnerable to political influence and lobbying, particularly in its development of cost-benefit analysis. This vulnerability arises from the high level of external scrutiny and the need to justify decisions in the face of competing interests.
Key concept: Pork Barrel Politics: This phrase epitomizes the political dynamics that have shaped the US Army Corps of Engineers and its approach to cost-benefit analysis. ‘Pork barrel’ refers to the practice of securing government funding for projects primarily to benefit a particular district or constituency, often with little regard for overall economic efficiency or national interest.
8. Objectivity and the Politics of Disciplines
The pressure to adopt standardized, quantitative methods in academic disciplines often stems from external pressures, particularly from interactions with politics, government regulation, and the legal system. Disciplines such as medicine, psychology, and education have seen a rise in statistical analysis and standardized testing, often as a way to address public concerns about objectivity and fairness, as well as to increase the credibility and authority of their research findings.
Key concept: ‘All science is measurement.’ This quote from Helmholtz, frequently invoked by Austin Bradford Hill, reflects the growing importance of quantitative methods in many scientific disciplines. It also highlights the central role that standardization of instruments and procedures plays in enabling the measurement of phenomena across different contexts and observers.
9. Is Science Made by Communities?
While the idea of scientific community is widely invoked, the pursuit of objectivity often reflects the weakness and vulnerability of these communities. Standardized procedures and a focus on quantitative methods can serve as alternatives to trust within scientific communities, allowing for agreement and communication across differences in background, skill, and perspective. But when communities become more secure, the emphasis on strict objectivity often becomes less urgent.
Key concept: ‘Interpretive communities.’ This concept from Stanley Fish’s literary theory emphasizes that meaning is not inherent in texts but is constructed by communities of readers who share certain interpretive strategies and assumptions. Applying this idea to science suggests that scientific knowledge, like literary meaning, is socially constructed and validated within specific disciplinary communities.
Essential Questions
1. Why is there a pursuit of objectivity?
The pursuit of objectivity in science, and even more in public affairs, is driven by distrust and a desire to replace personal judgment with impersonal rules. This has become especially prevalent in modern democracies and large bureaucratic organizations, where officials lack the traditional forms of legitimacy derived from inherited rank or personal charisma. Quantification and the appeal to numbers, or to seemingly neutral ‘facts,’ lends authority to officials who have very little of their own. This push for objectivity often reflects a desire for accountability and fairness, as well as efficiency and control. By appealing to numbers, decision-makers can appear to be making choices ‘by the numbers,’ shielding themselves from accusations of bias or arbitrariness.
2. What is quantification?
Quantification is a tool for standardization, making the local universal. The development of uniform standards and measures, often involving the collaboration of scientists, engineers, and government agencies, has been crucial for transforming local skills and knowledge into generally valid scientific principles. Standardization plays a similar role in social and economic life. By defining categories, establishing protocols for measurement, and imposing uniform accounting procedures, bureaucratic organizations can manage and control complex activities across vast distances and diverse populations. Standardization, thus, is essential for building large-scale organizations and for creating the modern information society.
3. How is quantification a technology of distance?
Quantification is a “technology of distance” because it minimizes the need for intimate knowledge and personal trust. Standardized numbers and procedures allow for communication across geographical and cultural boundaries without requiring face-to-face contact or reliance on local knowledge. This is essential for the operation of large-scale organizations like governments and corporations, and it has also been crucial for the development of a global scientific network. The highly structured language of mathematics, with its strict rules and formal procedures, facilitates communication and coordination among strangers, or at least among those who do not share a common culture or background.
4. Is quantification effective in establishing truth and validity?
Yes, but not simply because quantification offers a way to get at the ‘true nature of things.’ While some quantitative researchers, like Karl Pearson, have embraced a strong positivist faith in the power of numbers to reveal objective truths, I argue that the appeal of quantification in modern life is less about truth and more about trust, or rather the lack of it. In situations of distrust, where personal judgment and expertise are viewed with suspicion, standardized numbers and procedures offer a seemingly impartial alternative, lending legitimacy and authority to decisions. The quest for objectivity, then, is not just a scientific ideal but also a social and political one, shaped by context and by power relations.
1. Why is there a pursuit of objectivity?
The pursuit of objectivity in science, and even more in public affairs, is driven by distrust and a desire to replace personal judgment with impersonal rules. This has become especially prevalent in modern democracies and large bureaucratic organizations, where officials lack the traditional forms of legitimacy derived from inherited rank or personal charisma. Quantification and the appeal to numbers, or to seemingly neutral ‘facts,’ lends authority to officials who have very little of their own. This push for objectivity often reflects a desire for accountability and fairness, as well as efficiency and control. By appealing to numbers, decision-makers can appear to be making choices ‘by the numbers,’ shielding themselves from accusations of bias or arbitrariness.
2. What is quantification?
Quantification is a tool for standardization, making the local universal. The development of uniform standards and measures, often involving the collaboration of scientists, engineers, and government agencies, has been crucial for transforming local skills and knowledge into generally valid scientific principles. Standardization plays a similar role in social and economic life. By defining categories, establishing protocols for measurement, and imposing uniform accounting procedures, bureaucratic organizations can manage and control complex activities across vast distances and diverse populations. Standardization, thus, is essential for building large-scale organizations and for creating the modern information society.
3. How is quantification a technology of distance?
Quantification is a “technology of distance” because it minimizes the need for intimate knowledge and personal trust. Standardized numbers and procedures allow for communication across geographical and cultural boundaries without requiring face-to-face contact or reliance on local knowledge. This is essential for the operation of large-scale organizations like governments and corporations, and it has also been crucial for the development of a global scientific network. The highly structured language of mathematics, with its strict rules and formal procedures, facilitates communication and coordination among strangers, or at least among those who do not share a common culture or background.
4. Is quantification effective in establishing truth and validity?
Yes, but not simply because quantification offers a way to get at the ‘true nature of things.’ While some quantitative researchers, like Karl Pearson, have embraced a strong positivist faith in the power of numbers to reveal objective truths, I argue that the appeal of quantification in modern life is less about truth and more about trust, or rather the lack of it. In situations of distrust, where personal judgment and expertise are viewed with suspicion, standardized numbers and procedures offer a seemingly impartial alternative, lending legitimacy and authority to decisions. The quest for objectivity, then, is not just a scientific ideal but also a social and political one, shaped by context and by power relations.
Key Takeaways
1. Objectivity can mask bias
The pursuit of objectivity through quantification and standardized procedures can have unintended consequences, particularly when applied to complex social phenomena. Standardized measures can mask underlying biases or inequalities, creating the illusion of fairness while reinforcing existing power structures. A focus on easily measurable quantities can lead to a neglect of important qualitative factors, resulting in superficial analyses that miss the point.
Practical Application:
In designing an AI system for medical diagnosis, it is essential to recognize that algorithms trained on existing datasets might perpetuate or amplify biases present in those datasets. A focus on purely statistical measures of performance could lead to the development of a system that is highly accurate overall but performs poorly for certain demographic groups. Hence, ongoing evaluation and critical engagement with the social and ethical implications of AI technologies are crucial.
2. Context matters in interpretation
Numbers don’t speak for themselves. The meaning and validity of quantitative data always depend on context and on the social and administrative processes through which they are generated. Statistical categories, economic measures, and other seemingly objective data points are often constructed through negotiation, compromise, and the exercise of discretion. Ignoring the social and political dimensions of quantification can lead to a distorted understanding of its role and influence.
Practical Application:
If tasked with developing an AI system for loan applications, simply relying on historical data may perpetuate past discriminatory lending practices. Moreover, a focus on easily quantifiable factors like credit scores could disadvantage applicants from marginalized communities. Hence, qualitative information and human oversight may be necessary to ensure fairness and avoid perpetuating bias.
3. Quantification narrows vision
While standardized procedures and quantitative methods can be powerful tools for managing complexity, they can also lead to a narrowing of vision and a neglect of important qualitative factors. A focus on efficiency and measurable outcomes, whether in business management, public administration, or scientific research, can create an ‘accounting ideal’ that prioritizes what can be easily quantified over more nuanced considerations.
Practical Application:
A team developing an AI-powered educational tool might be tempted to prioritize standardized test scores as the primary measure of student learning. But a singular focus on quantifiable metrics risks neglecting the complex, multi-faceted nature of education and may lead to teaching to the test or other undesirable outcomes. Hence, it’s important to incorporate qualitative assessments and consider the broader social and ethical implications of the technology.
1. Objectivity can mask bias
The pursuit of objectivity through quantification and standardized procedures can have unintended consequences, particularly when applied to complex social phenomena. Standardized measures can mask underlying biases or inequalities, creating the illusion of fairness while reinforcing existing power structures. A focus on easily measurable quantities can lead to a neglect of important qualitative factors, resulting in superficial analyses that miss the point.
Practical Application:
In designing an AI system for medical diagnosis, it is essential to recognize that algorithms trained on existing datasets might perpetuate or amplify biases present in those datasets. A focus on purely statistical measures of performance could lead to the development of a system that is highly accurate overall but performs poorly for certain demographic groups. Hence, ongoing evaluation and critical engagement with the social and ethical implications of AI technologies are crucial.
2. Context matters in interpretation
Numbers don’t speak for themselves. The meaning and validity of quantitative data always depend on context and on the social and administrative processes through which they are generated. Statistical categories, economic measures, and other seemingly objective data points are often constructed through negotiation, compromise, and the exercise of discretion. Ignoring the social and political dimensions of quantification can lead to a distorted understanding of its role and influence.
Practical Application:
If tasked with developing an AI system for loan applications, simply relying on historical data may perpetuate past discriminatory lending practices. Moreover, a focus on easily quantifiable factors like credit scores could disadvantage applicants from marginalized communities. Hence, qualitative information and human oversight may be necessary to ensure fairness and avoid perpetuating bias.
3. Quantification narrows vision
While standardized procedures and quantitative methods can be powerful tools for managing complexity, they can also lead to a narrowing of vision and a neglect of important qualitative factors. A focus on efficiency and measurable outcomes, whether in business management, public administration, or scientific research, can create an ‘accounting ideal’ that prioritizes what can be easily quantified over more nuanced considerations.
Practical Application:
A team developing an AI-powered educational tool might be tempted to prioritize standardized test scores as the primary measure of student learning. But a singular focus on quantifiable metrics risks neglecting the complex, multi-faceted nature of education and may lead to teaching to the test or other undesirable outcomes. Hence, it’s important to incorporate qualitative assessments and consider the broader social and ethical implications of the technology.
Memorable Quotes
Chapter 1. 13
A scientific truth has little standing until it becomes a collective product.
Chapter 1. 21
The growth of science has to a large degree involved the replacement of nature by human technologies.
Chapter 3. 74
Precision has been valued as a sign of diligence, skill, and impersonality.
Chapter 3. 77
This form of economics was ideally suited to become an economics of measurement.
Preface to the New Edition. 8
“This is based on the data,” the managers explain, echoing all those scientists who use ‘data-driven’ as the standard of legitimate knowledge.
Chapter 1. 13
A scientific truth has little standing until it becomes a collective product.
Chapter 1. 21
The growth of science has to a large degree involved the replacement of nature by human technologies.
Chapter 3. 74
Precision has been valued as a sign of diligence, skill, and impersonality.
Chapter 3. 77
This form of economics was ideally suited to become an economics of measurement.
Preface to the New Edition. 8
“This is based on the data,” the managers explain, echoing all those scientists who use ‘data-driven’ as the standard of legitimate knowledge.
Comparative Analysis
My work resonates with several other scholars who examine the interplay of knowledge, power, and social structures. Michel Foucault’s work on power and knowledge, particularly his concept of ‘governmentality’, is highly relevant to my argument. I also draw upon the work of Steven Shapin and Simon Schaffer in Leviathan and the Air-Pump, which examines the social construction of scientific knowledge in 17th-century England. Like Shapin and Schaffer, I emphasize the importance of skills, communities, and face-to-face interactions in science. My analysis, however, extends beyond the laboratory to consider the broader social and political contexts of quantification. While Karin Knorr-Cetina’s work on the ‘manufacture of knowledge’ shares my focus on laboratory practices, I am less interested in epistemological questions than in the practical uses of quantification in administrative and political settings. I agree with Bruno Latour’s emphasis on the close connection between science and technology (‘technoscience’), but I move beyond his actor-network theory to examine the specific ways in which quantification functions as a technology of trust and distance. Finally, while many scholars, especially those writing during the “science wars,” have taken strong positions for or against scientific objectivity, I offer a more nuanced perspective, emphasizing both its power and its limitations as a tool for resolving disputes and making decisions.
Reflection
This book is a timely reminder that numbers, while seemingly neutral and objective, are always embedded in social and political contexts. The quest for objectivity through quantification is not simply a scientific or technical endeavor but a cultural phenomenon with profound implications for how we organize our institutions, make decisions, and understand the world around us. The book’s analysis of historical cases provides valuable insights into the complex interplay of knowledge, power, and social structures. While the focus on historical examples, mostly from the 19th and mid-20th centuries, might limit its direct applicability to contemporary issues, it provides a framework for understanding the deeper roots of the current obsession with data, metrics, and algorithms.
While the book avoids taking a strong position for or against quantification, it encourages a healthy skepticism towards the naive faith in ‘data-driven’ decision-making that has become so prevalent in recent years. It reminds us that even the most sophisticated quantitative tools are subject to interpretation, manipulation, and bias, and that expert judgment, nuanced reasoning, and social and ethical considerations remain essential for navigating the complexities of modern life. Especially relevant in the age of AI, my work reminds us that numbers don’t tell the whole story. Human designers and users are always part of the equation.
Flashcards
How does one achieve objectivity in measurement?
Standardization is the key, involving the creation of uniform standards and measures, administrative routines, and other technologies of trust. This process often involves putting aside deep meanings and convictions in favor of compromise and convention, not necessarily involving any kind of implicit trust, but more generally skepticism, reluctance, and hesitation.
What role does quantification play in modern institutions?
Quantification is a tool for building trust and exercising authority in situations where personal judgment is viewed with suspicion. It allows for standardized, seemingly objective comparisons that can be used to make decisions and justify policies without appearing to rely on subjective preferences or biases.
What are some potential social and political consequences of quantification?
While often presented as neutral and objective, quantitative technologies like standardized tests and cost-benefit analysis can have powerful social and political effects. They can shape perceptions, reinforce existing power structures, and create new forms of social control.
How has the culture of quantification changed over time?
In pre-modern societies, measurement was often a matter of local custom and negotiation, with units varying by region, material, and even social status. The shift toward standardized, universal measures was closely linked to the rise of modern states, the growth of commerce and industry, and the development of scientific disciplines.
What factors drove the standardization of cost-benefit analysis at the U.S. Army Corps of Engineers?
The most effective pressures for standardization in cost-benefit analysis came not from within the Corps, but from its rivals and opponents, including utility companies, railroads, and other government agencies. These groups had an incentive to scrutinize the Corps’ methods and push for greater uniformity as a way to limit its discretion and challenge its authority.
What drove the shift to standardized units like the metric system in the 18th and 19th centuries?
Many factors, including the rise of modern states, growth of commerce and industry, and the development of scientific disciplines contributed to the shift towards standardization.
What are some of the limitations of quantification?
While quantification offers a seemingly objective basis for decisions, this often comes at the cost of ignoring or oversimplifying important qualitative factors and nuances of meaning.
How does one achieve objectivity in measurement?
Standardization is the key, involving the creation of uniform standards and measures, administrative routines, and other technologies of trust. This process often involves putting aside deep meanings and convictions in favor of compromise and convention, not necessarily involving any kind of implicit trust, but more generally skepticism, reluctance, and hesitation.
What role does quantification play in modern institutions?
Quantification is a tool for building trust and exercising authority in situations where personal judgment is viewed with suspicion. It allows for standardized, seemingly objective comparisons that can be used to make decisions and justify policies without appearing to rely on subjective preferences or biases.
What are some potential social and political consequences of quantification?
While often presented as neutral and objective, quantitative technologies like standardized tests and cost-benefit analysis can have powerful social and political effects. They can shape perceptions, reinforce existing power structures, and create new forms of social control.
How has the culture of quantification changed over time?
In pre-modern societies, measurement was often a matter of local custom and negotiation, with units varying by region, material, and even social status. The shift toward standardized, universal measures was closely linked to the rise of modern states, the growth of commerce and industry, and the development of scientific disciplines.
What factors drove the standardization of cost-benefit analysis at the U.S. Army Corps of Engineers?
The most effective pressures for standardization in cost-benefit analysis came not from within the Corps, but from its rivals and opponents, including utility companies, railroads, and other government agencies. These groups had an incentive to scrutinize the Corps’ methods and push for greater uniformity as a way to limit its discretion and challenge its authority.
What drove the shift to standardized units like the metric system in the 18th and 19th centuries?
Many factors, including the rise of modern states, growth of commerce and industry, and the development of scientific disciplines contributed to the shift towards standardization.
What are some of the limitations of quantification?
While quantification offers a seemingly objective basis for decisions, this often comes at the cost of ignoring or oversimplifying important qualitative factors and nuances of meaning.